Velocity control for an unmanned aerial vehicle

ABSTRACT

Systems and methods for controlling an unmanned aerial vehicle within an environment are provided. In one aspect, a system comprises one or more sensors carried on the unmanned aerial vehicle and configured to receive sensor data of the environment and one or more processors. The one or more processors may be individually or collectively configured to: determine, based on the sensor data, an environmental complexity factor representative of an obstacle density for the environment; determine, based on the environmental complexity factor, one or more operating rules for the unmanned aerial vehicle; receive a signal indicating a desired movement of the unmanned aerial vehicle; and cause the unmanned aerial vehicle to move in accordance with the signal while complying with the one or more operating rules.

CROSS-REFERENCE

This application is a continuation of U.S. patent application Ser. No.14/801,599, filed Jul. 16, 2015, which is a continuation ofInternational Application No. PCT/CN2014/086005, filed Sep. 5, 2014, thecontents of which are hereby incorporated by reference in theirentireties.

BACKGROUND

Unmanned vehicles such as unmanned aerial vehicles can be used forperforming surveillance, reconnaissance, and exploration tasks in a widevariety of environments for military and civilian applications. Anunmanned aerial vehicle may be manually controlled by a remote user, ormay operate in a semi-autonomous or fully autonomous manner. Variouscontrol schemes can be implemented to enable operation of an unmannedaerial vehicle within an environment with varying degrees of autonomy.

However, existing control schemes for unmanned aerial vehicles may notbe optimal in some instances. For example, an unmanned aerial vehiclemay operate in environments in which obstacles are present (e.g.,buildings, trees, human beings, other aerial objects). Existing controlschemes may rely upon the user's own judgment to determine safeoperating parameters for the unmanned aerial vehicle in order tominimize the probability of collisions with obstacles. This may bechallenging for inexperienced users or in situations where the usercannot easily see the environment surrounding the unmanned aerialvehicle (e.g., when the unmanned aerial vehicle is relatively far fromthe user).

SUMMARY

Improved control schemes for improving the safety of unmanned aerialvehicles are needed. The present disclosure provides systems and methodsfor automatically determining operating rules for an unmanned aerialvehicle. The operating rules can pertain to any suitable aspect of UAVoperation, such as velocity, acceleration, position, or orientation. Insome embodiments, the operating rules are automatically determinedduring flight using data obtained from various sensors. The sensor datacan be analyzed to determine the obstacle density of the environment inorder to select appropriate operating rules for controlling the unmannedaerial vehicle, thereby reducing the likelihood of collisions withobstacles.

Thus, in one aspect, a system for controlling an unmanned aerial vehiclewithin an environment is provided. The system comprises one or moresensors carried on the unmanned aerial vehicle and configured to receivesensor data of the environment and one or more processors. The one ormore processors may be individually or collectively configured to:determine, based on the sensor data, an environmental complexity factorrepresentative of an obstacle density for the environment; determine,based on the environmental complexity factor, one or more operatingrules for the unmanned aerial vehicle; receive a signal indicating adesired movement of the unmanned aerial vehicle; and cause the unmannedaerial vehicle to move in accordance with the signal while complyingwith the one or more operating rules.

In some embodiments, the one or more sensors comprise a vision sensor,such as a stereovision sensor. The one or more sensors can comprise alidar sensor or an ultrasonic sensor. The one or more sensors cancomprise a plurality of different sensor types. The sensor data obtainedby the one or more sensors may be indicative of an obstacle density forthe environment.

In some embodiments, the one or more processors are carried by theunmanned aerial vehicle. The environmental complexity factor can bedetermined based on a three-dimensional digital representation of theenvironment generated using the sensor data. The three-dimensionaldigital representation can comprise a three-dimensional point cloud oran occupancy grid.

The operating rules can be determined in any suitable manner. Forinstance, the one or more operating rules can be configured to preventcollisions between the unmanned aerial vehicle and obstacles in theenvironment. The one or more operating rules can be determined based onpreviously obtained flight data. In some embodiments, the one or moreoperating rules comprise one or more velocity rules. The one or morevelocity rules can be determined using a first-in-first-out (FIFO) queueof previously determined velocity rules. The one or more velocity rulescan be determined based on a minimum braking distance for the unmannedaerial vehicle. In some embodiments, the one or more velocity rulescomprise a velocity limit for the unmanned aerial vehicle.

In some embodiments, the signal comprises a user input command.

In some embodiments, the one or more operating rules comprise one ormore attitude rules. Alternatively or in combination, the one or moreoperating rules can comprise one or more altitude rules.

In another aspect, a method for controlling an unmanned aerial vehiclewithin an environment is provided. The method comprises: receivingsensor data of the environment from one or more sensors carried on theunmanned aerial vehicle; determining, based on the sensor data and withaid of a processor, an environmental complexity factor representative ofan obstacle density for the environment; determining, based on theenvironmental complexity factor and with aid of the processor, one ormore operating rules for the unmanned aerial vehicle; receiving a signalindicating a desired movement of the unmanned aerial vehicle; andcausing the unmanned aerial vehicle to move in accordance with thesignal while complying with the one or more operating rules.

In another aspect, a system for controlling an unmanned aerial vehiclewithin an environment is provided. The system comprises one or moresensors carried on the unmanned aerial vehicle and configured to receivesensor data of the environment and one or more processors. The one ormore processors may be individually or collectively configured to:determine, based on the sensor data, a first set of operating rules forthe unmanned aerial vehicle; receive user input indicating a second setof operating rules for the unmanned aerial vehicle; select one of thefirst or second sets of operating rules to be used to control theunmanned aerial vehicle; receive a signal indicating a desired movementof the unmanned aerial vehicle; and cause the unmanned aerial vehicle tomove in accordance with the signal while complying with the selected oneof the first or second sets of operating rules.

In some embodiments, the one or more sensors comprise a vision sensor,such as a stereovision sensor. The one or more sensors can comprise alidar sensor or an ultrasonic sensor. The one or more sensors cancomprise a plurality of different sensor types. The sensor data obtainedby the one or more sensors may be indicative of an obstacle density forthe environment. The one or more processors can be carried by theunmanned aerial vehicle.

In some embodiments, the user input can be received from a remoteterminal.

The operating rules can be configured as desired. For instance, thefirst set of operating rules can be configured to prevent collisionsbetween the unmanned aerial vehicle and obstacles in the environment. Insome embodiments, the first set of operating rules comprises a first setof velocity rules and the second set of operating rules comprises asecond set of velocity rules. The first set of velocity rules can bedetermined based on a minimum braking distance for the unmanned aerialvehicle. The second set of velocity rules can be determined based on aflight mode selected by the user from a plurality of different flightmodes. The plurality of different flight modes can comprise a lowvelocity flight mode, an intermediate velocity flight mode, and a highvelocity flight mode. In some embodiments, the first set of velocityrules comprises a first velocity limit for the unmanned aerial vehicleand the second set of velocity rules comprises a second velocity limitfor the unmanned aerial vehicle. The one or more processors may selectone of the first or second set of velocity rules by selecting thesmaller of the first and second velocity limits.

In some embodiments, the signal comprises a user input command.

In some embodiments, the first and second sets of operating rules eachcomprise a set of attitude rules. Alternatively or in combination, thefirst and second sets of operating rules can each comprise a set ofaltitude rules.

In another aspect, a method for controlling an unmanned aerial vehiclewithin an environment is provided. The method comprises: receivingsensor data of the environment from one or more sensors carried on theunmanned aerial vehicle; determining, based on the sensor data and withaid of a processor, a first set of operating rules for the unmannedaerial vehicle; receiving user input indicating a second set ofoperating rules for the unmanned aerial vehicle; selecting one of thefirst or second sets of operating rules to be used to control theunmanned aerial vehicle; receiving a signal indicating a desiredmovement of the unmanned aerial vehicle; and causing the unmanned aerialvehicle to move in accordance with the signal while complying with theselected one of the first or second sets of operating rules.

In another aspect, a system for controlling an unmanned aerial vehiclewithin an environment is provided. The system comprises one or moresensors carried on the unmanned aerial vehicle; and one or moreprocessors. The one or more processors may be individually orcollectively configured to: determine, using the one or more sensors, anenvironmental complexity factor representative of an obstacle densityfor the environment; determine, based on the environmental complexityfactor and with aid of the processor, a first set of operating rules forthe unmanned aerial vehicle; detect, using the one or more sensors, achange in the environmental complexity factor corresponding to a changein the obstacle density for the environment; and modify the first set ofoperating rules based on the change in the environmental complexityfactor to provide a second set of operating rules for the unmannedaerial vehicle.

In some embodiments, the one or more sensors comprise a vision sensor,such as a stereovision sensor. The one or more sensors can comprise alidar sensor or an ultrasonic sensor. The one or more sensors cancomprise a plurality of different sensor types. The sensor data obtainedby the one or more sensors may be indicative of an obstacle density forthe environment.

In some embodiments, the one or more processors are carried by theunmanned aerial vehicle. The environmental complexity factor can bedetermined based on a three-dimensional digital representation of theenvironment generated using the sensor data. The three-dimensionaldigital representation can comprise a three-dimensional point cloud oran occupancy grid.

In some embodiments, at least one of the first and second sets ofoperating rules can be configured to prevent collisions between theunmanned aerial vehicle and obstacles in the environment. The first setof operating rules can comprise a first set of velocity rules and thesecond set of operating rules can comprise a second set of velocityrules. At least one of the first and second sets of velocity rules canbe determined based on a minimum braking distance for the unmannedaerial vehicle. Optionally, the first and second sets of velocity rulesmay each comprise a first velocity limit for the unmanned aerialvehicle. In some embodiments, the change in the environmental complexityfactor corresponds to a decrease in the obstacle density, and thevelocity limit of the second set of velocity rules is greater than thevelocity limit of the first set of velocity rules.

In some embodiments, the first and second sets of operating rules eachcomprise a set of attitude rules. Alternatively or in combination, thefirst and second sets of operating rules can each comprise a set ofaltitude rules.

In another aspect, a method for controlling an unmanned aerial vehiclewithin an environment is provided. The method comprises: determining,using one or more sensors carried by the unmanned aerial vehicle andwith aid of a processor, an environmental complexity factorrepresentative of an obstacle density for the environment; determining,based on the environmental complexity factor and with aid of theprocessor, a first set of operating rules for the unmanned aerialvehicle; detecting, using the one or more sensors, a change in theenvironmental complexity factor corresponding to a change in theobstacle density for the environment; and modifying the first set ofoperating rules based on the change in the environmental complexityfactor to provide a second set of operating rules for the unmannedaerial vehicle.

It shall be understood that different aspects of the invention can beappreciated individually, collectively, or in combination with eachother. Various aspects of the invention described herein may be appliedto any of the particular applications set forth below or for any othertypes of movable objects. Any description herein of an aerial vehiclemay apply to and be used for any movable object, such as any vehicle.Additionally, the systems, devices, and methods disclosed herein in thecontext of aerial motion (e.g., flight) may also be applied in thecontext of other types of motion, such as movement on the ground or onwater, underwater motion, or motion in space. Furthermore, anydescription herein of a rotor or rotor assembly may apply to and be usedfor any propulsion system, device, or mechanism configured to generate apropulsive force by rotation (e.g., propellers, wheels, axles).

Other objects and features of the present invention will become apparentby a review of the specification, claims, and appended figures.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

FIG. 1A illustrates an unmanned aerial vehicle operating in an outdoorenvironment, in accordance with embodiments;

FIG. 1B illustrates an unmanned aerial vehicle operating in an indoorenvironment, in accordance with embodiments;

FIG. 2 illustrates a method for determining operating rules forcontrolling an unmanned aerial vehicle, in accordance with embodiments;

FIG. 3 illustrates a method for controlling an unmanned aerial vehicle,in accordance with embodiments;

FIG. 4 illustrates a disparity map generated from sensor data of anenvironment, in accordance with embodiments;

FIG. 5 illustrates a histogram indicative of obstacle density in anenvironment, in accordance with embodiments;

FIG. 6 illustrates a first-in-first-out queue for determining velocityrules, in accordance with embodiments;

FIG. 7 illustrates a method for controlling a UAV, in accordance withembodiments;

FIG. 8 illustrates a remote controller for controlling an unmannedaerial vehicle, in accordance with embodiments;

FIG. 9 illustrates an unmanned aerial vehicle, in accordance withembodiments;

FIG. 10 illustrates a movable object including a carrier and a payload,in accordance with embodiments; and

FIG. 11 illustrates a system for controlling a movable object, inaccordance with embodiments.

DETAILED DESCRIPTION

The present disclosure provides improved systems and methods fordetermining operating rules for operating an unmanned aerial vehicle(UAV). The UAV can carry one or more sensors used to obtain data of thesurrounding environment, and this data can subsequently be processed todetect the degree to which obstacles and other potential safety hazardsare present in the surrounding environment, which may be referred toherein as an “environmental complexity factor.” The environmentalcomplexity factor can be used to determine a set of operating rules(e.g., velocity rules such as velocity limits or velocity ranges) to beobserved while operating the UAV within the environment. For example,the maximum velocity limit for the UAV may be relatively low when theUAV is operating within a “complex” environment (an environment with ahigh obstacle density, such as an indoor, urban, or low altitudeenvironment), thereby reducing the risk of accidental collisions.Conversely, the maximum velocity limit for the UAV may be relativelyhigh when operating within a less complex environment in whichcollisions are unlikely to occur (an environment having a low obstacledensity, such as a high altitude environment). Advantageously, theembodiments described herein can be used to automatically anddynamically optimize the operating parameters for the UAV based on thecurrent environmental context, thereby enhancing the safety, ease ofuse, and adaptability of the UAV.

The UAVs described herein can be operated autonomously (e.g., by asuitable computing system such as an onboard controller),semi-autonomously, or manually (e.g., by a human user). The UAV canreceive commands from a suitable entity (e.g., human user or autonomouscontroller) and respond to such commands by performing one or moreactions. For example, the UAV can be controlled to take off from theground, move within the air (e.g., with up to three degrees of freedomin translation and up to three degrees of freedom in rotation), hoverwithin the air, land on the ground, and so on. As another example, theUAV can be controlled to move at a specified velocity and/oracceleration (e.g., with up to three degrees of freedom in translationand up to three degrees of freedom in rotation). The UAV may have amaximum horizontal velocity of approximately 13 m/s, or within a rangefrom 10 m/s to 15 m/s. The UAV may have a maximum vertical velocity ofapproximately 6 m/s, or within a range from 5 m/s to 10 m/s. The UAV mayhave a maximum translational acceleration of approximately 6.8 m/s², orwithin a range from 5 m/s² to 10 m/s². In some embodiments,translational acceleration of the UAV may cause the UAV to assume acorresponding attitude. Accordingly, the maximum translationalacceleration of the UAV may be constrained by the maximum attitude. Insome embodiments, the maximum attitude angle for the UAV (relative tothe vertical axis of the UAV) may be approximately 35°, or within arange from 25° to 45°. Attitude may be used herein to refer to the rolland/or pitch angle of the UAV.

Turning now the drawings, FIG. 1A illustrates a UAV 102 operating in anoutdoor environment 100, in accordance with embodiments. The outdoorenvironment 100 may be an urban, suburban, or rural setting, or anyother environment that is not at least partially within a building. TheUAV 102 may be operated relatively close to the ground 104 (e.g., lowaltitude) or relatively far from the ground 104 (e.g., high altitude).For example, a UAV 102 operating less than or equal to approximately 10m from the ground may be considered to be at low altitude, while a UAV102 operating at greater than or equal approximately 10 m from theground may be considered to be at high altitude.

In some embodiments, the outdoor environment 100 includes one or moreobstacles 108 a-d. An obstacle may include any object or entity that mayobstruct the movement of the UAV 102. Some obstacles may be situated onthe ground 104 (e.g., obstacles 108 a, 108 d), such as buildings, groundvehicles (e.g., cars, motorcycles, trucks, bicycles), human beings,animals, plants (e.g., trees, bushes), and other manmade or naturalstructures. Some obstacles may be in contact with and/or supported bythe ground 104, water, manmade structures, or natural structures.Alternatively, some obstacles may be wholly located in the air 106(e.g., obstacles 108 b, 108 c), including aerial vehicles (e.g.,airplanes, helicopters, hot air balloons, other UAVs) or birds. Aerialobstacles may not be supported by the ground 104, or by water, or by anynatural or manmade structures. An obstacle located on the ground 104 mayinclude portions that extend substantially into the air 106 (e.g., tallstructures such as towers, skyscrapers, lamp posts, radio towers, powerlines, trees, etc.).

FIG. 1B illustrates a UAV 152 operating in an indoor environment 150, inaccordance with embodiments. The indoor environment 150 is within theinterior of a building 154 having a floor 156, one or more walls 158,and/or a ceiling or roof 160. Exemplary buildings include residential,commercial, or industrial buildings such as houses, apartments, offices,manufacturing facilities, storage facilities, and so on. The interior ofthe building 154 may be completely enclosed by the floor 156, walls 158,and ceiling 160 such that the UAV 152 is constrained to the interiorspace. Conversely, at least one of the floor 156, walls 158, or ceiling160 may be absent, thereby enabling the UAV 152 to fly from inside tooutside, or vice-versa. Alternatively or in combination, one or moreapertures 164 may be formed in the floor 156, walls 158, or ceiling 160(e.g., a door, window, skylight).

Similar to the outdoor environment 100, the indoor environment 150 caninclude one or more obstacles 162 a-d. Some obstacles may be situated onthe floor 156 (e.g., obstacle 162 a), such as furniture, appliances,human beings, animals, plants, and other manmade or natural objects.Conversely, some obstacles may be located in the air (e.g., obstacle 162b), such as birds or other UAVs. Some obstacles in the indoorenvironment 150 can be supported by other structures or objects.Obstacles may also be attached to the ceiling 160 (e.g., obstacle 162c), such as light fixtures, ceiling fans, beams, or otherceiling-mounted appliances or structures. In some embodiments, obstaclesmay be attached to the walls 158 (e.g., obstacle 162 d), such as lightfixtures, shelves, cabinets, and other wall-mounted appliances orstructures. Notably, the structural components of the building 154 canalso be considered to be obstacles, including the floor 156, walls 158,and ceiling 160.

The obstacles described herein may be substantially stationary (e.g.,buildings, plants, structures) or substantially mobile (e.g., humanbeings, animals, vehicles, or other objects capable of movement). Someobstacles may include a combination of stationary and mobile components(e.g., a windmill). Mobile obstacles or obstacle components may moveaccording to a predetermined or predictable path or pattern. Forexample, the movement of a car may be relatively predictable (e.g.,according to the shape of the road). Alternatively, some mobileobstacles or obstacle components may move along random or otherwiseunpredictable trajectories. For example, a living being such as ananimal may move in a relatively unpredictable manner.

The safety risk associated with operating a UAV within a particularenvironment may be related to the amount and types of obstacles withinthe environment. Different types of environments may be associated withdifferent amounts and types of obstacles. For example, a high altitudeenvironment may have few or no obstacles. In contrast, an indoorenvironment or a low altitude environment may have more obstacles. Sometypes of low altitude, outdoor environments (e.g., fields and otherflat, open spaces) may have fewer obstacles than other types (e.g.,urban settings and other highly populated areas, forests). Accordingly,a UAV operating within an environment with a high obstacle density maybe exposed to an increased risk of collisions, near-misses, or othersafety incidents. Conversely, UAV operation within a low obstacledensity environment may be relatively safe. Mobile obstacles may pose anincreased risk compared to stationary obstacles, as mobile obstacles maycollide with or obstruct the UAV independently of any action taken bythe UAV.

A UAV can be operated within various environments in accordance with oneor more operating rules. In some embodiments, the one or more operatingrules may provide ranges, limits, values, and the like for one or moreaspects of the state of the UAV (e.g., altitude, latitude, longitude,roll, pitch, yaw, translational velocity, angular velocity,translational acceleration, angular acceleration, etc.). As anotherexample, one or more operating rules may provide algorithms, logic,processes, and the like for implementing one or more UAV functionalities(e.g., collision avoidance, fault detection, safety mechanisms,navigation, mapping, data collection, data processing, etc.).Alternatively or in combination, exemplary operating rules may providealgorithms, logic, processes, and the like for controlling one or moreaspects of UAV behavior (e.g., response to user commands, response todetected obstacles, response to errors or malfunctions, autonomous orsemi-autonomous operation, etc.).

For example, in some embodiments, the operating rules can include one ormore velocity rules for the UAV. Exemplary velocity rules that may beapplied to control the operation of a UAV may include a velocity limit(e.g., a maximum or minimum velocity value), a velocity range, avelocity value, or suitable combinations thereof. The velocity rules fora UAV may be related to the maximum velocity of the UAV. For instance,the velocity limit or velocity value can be approximately 10%, 20%, 30%,40%, 50%, 60%, 70%, 80%, 90% or 100% of the maximum velocity. Thevelocity range can be between any two of the following values: 10%, 20%,30%, 40%, 50%, 60%, 70%, 80%, 90% or 100% of the maximum velocity. Thevelocity rules may apply to all directions of motion (e.g., up/down,left/right, forwards/backwards, roll, pitch, yaw). Alternatively, thevelocity rules may apply only to movement along certain directions.Furthermore, the velocity rules may be the same for all directions, ormay differ for some directions. For example, the velocity rules fortranslational movements may be different than the velocity rules forrotational movements.

Alternatively or in combination, other types of operating rules can alsobe used, such as rules providing ranges, limits, and/or values for UAVattitude, altitude, acceleration, and the like. Any description hereinpertaining to velocity rules can also be applied to other types ofoperating rules, and vice-versa.

Optionally, a set of operating rules may be associated with a flightmode for the UAV. The term “flight mode” may be used herein to refer toa control scheme for operating a UAV. Various types of flight modes canbe used to control a UAV, and each flight mode can include a differentset of operating rules. In some embodiments, a flight mode may bedefined based on the extent of user control provided by the flight mode(e.g., “free” or “manual”, “fully autonomous,” or “semi-autonomous”flight modes), the intended environment type for using the flight mode(e.g., “low altitude,” “high altitude,” “indoor,” “outdoor,” “longrange,” or “short range,” flight modes), the operating rules of theflight mode (e.g., “low velocity,” “intermediate velocity,” or “highvelocity” flight modes), or suitable combinations thereof. Anydescription herein pertaining to determination of one or more operatingrules can also be applied to determination of a flight mode associatedwith the one or more operating rules, and vice-versa. Any suitablenumber and combination of flight modes can be used, such as one, two,three, four, five, or more different flight modes, each corresponding toa respective set of operating rules. The appropriate operating rules foreach flight mode may be determined in any suitable manner, e.g., basedon analysis of previous flight data, machine learning, feedback fromtest users, etc.

The operating rules described herein can be used to improve variousaspects of UAV operation. For example, at least some of the operatingrules can be configured to reduce the safety risk associated with usinga UAV. The optimal operating rules for the UAV may vary based on thecurrent environmental conditions, such as the amount and types ofobstacles present within the environment. Accordingly, suitable methodsmay be implemented to assess the environment surrounding the UAV so asto determine an appropriate set of operating rules.

FIG. 2 illustrates a method 200 for determining operating rules forcontrolling a UAV within an environment, in accordance with embodiments.The method 200, as with all methods presented herein, can be practicedusing any embodiment of the systems and devices described herein. Forexample, one or more steps of the method 200 can be performed by one ormore processors, acting individually or collectively. Some of theprocessors may be carried by the UAV (e.g., on-board processors).Alternatively or in combination, some of the processors may be incommunication with the UAV from a remote location (e.g., a remotecomputing system or device). In some embodiments, a remote device can bea remote controller that accepts one or more user inputs to control oneor more components of another device (e.g., the UAV or portions thereof,a display unit, or other external devices), as described in furtherdetail herein. The method 200 can be performed in a fully automatedmanner without requiring any user input or manual intervention. In someembodiments, the steps of the method 200 are performed in real-timeduring operation of the UAV, thereby providing real-time context-basedadjustment of UAV operating rules.

In step 210, sensor data of the environment is received from one or moresensors. Any sensor suitable for collecting environmental informationcan be used, including location sensors (e.g., global positioning system(GPS) sensors, mobile device transmitters enabling locationtriangulation), vision sensors (e.g., imaging devices capable ofdetecting visible, infrared, or ultraviolet light, such as cameras),proximity sensors (e.g., ultrasonic sensors, lidar, time-of-flightcameras), inertial sensors (e.g., accelerometers, gyroscopes, inertialmeasurement units (IMUs)), altitude sensors, pressure sensors (e.g.,barometers), audio sensors (e.g., microphones) or field sensors (e.g.,magnetometers, electromagnetic sensors). Any suitable number andcombination of sensors can be used, such as one, two, three, four, five,or more sensors. Optionally, the data can be received from sensors ofdifferent types (e.g., two, three, four, five, or more types). Sensorsof different types may measure different types of signals or information(e.g., position, orientation, velocity, acceleration, proximity,pressure, etc.) and/or utilize different types of measurement techniquesto obtain data. For instance, the sensors may include any suitablecombination of active sensors (e.g., sensors that generate and measureenergy from their own source) and passive sensors (e.g., sensors thatdetect available energy).

The sensor data may provide various types of environmental information.For example, the sensor data may be indicative of an environment type,such as an indoor environment, outdoor environment, low altitudeenvironment, or high altitude environment. The sensor data may alsoprovide information regarding current environmental conditions,including weather (e.g., clear, rainy, snowing), visibility conditions,wind speed, time of day, and so on. Furthermore, the environmentalinformation collected by the sensors may include information regardingthe obstacles in the environment, such as the number of obstacles, thevolume or percentage of space occupied by obstacles, the volume orpercentage of space within a certain proximity to the UAV occupied byobstacles, the volume or percentage of space unobstructed by obstacles,the volume or percentage of space within a certain proximity to the UAVunobstructed by obstacles, the proximity of obstacles to the UAV, theobstacle density (e.g., number of obstacles per unit space), the typesof obstacles (e.g., stationary or mobile), the spatial disposition ofobstacles (e.g., position, orientation), the motion of obstacles (e.g.,velocity, acceleration), and so on.

In some embodiments, at least some of the sensors may be configured toprovide data regarding a state of the UAV. The state informationprovided by a sensor can include information regarding a spatialdisposition of the UAV (e.g., position, orientation). The stateinformation can also include information regarding motion of the UAV(e.g., translational velocity, translation acceleration, angularvelocity, angular acceleration, etc.). A sensor can be configured, forinstance, to determine a spatial disposition and/or motion of the UAVwith respect to up to six degrees of freedom (e.g., three degrees offreedom in position and/or translation, three degrees of freedom inorientation and/or rotation). The state information may be providedrelative to a global reference frame or relative to the reference frameof another entity.

The sensors described herein can be carried by the UAV. A sensor can besituated on any suitable portion of the UAV, such as above, underneath,on the side(s) of, or within a vehicle body of the UAV. Some sensors canbe mechanically coupled to the UAV such that the spatial dispositionand/or motion of the UAV correspond to the spatial disposition and/ormotion of the sensors. The sensor can be coupled to the UAV via a rigidcoupling, such that the sensor does not move relative to the portion ofthe UAV to which it is attached. Alternatively, the coupling between thesensor and the UAV can permit movement of the sensor relative to theUAV. The coupling can be a permanent coupling or non-permanent (e.g.,releasable) coupling. Suitable coupling methods can include adhesives,bonding, welding, and/or fasteners (e.g., screws, nails, pins, etc.).Optionally, the sensor can be integrally formed with a portion of theUAV. Furthermore, the sensor can be electrically coupled with a portionof the UAV (e.g., processing unit, control system, data storage) so asto enable the data collected by the sensor to be used for variousfunctions of the UAV (e.g., navigation, control, propulsion,communication with a user or other device, etc.), such as theembodiments discussed herein.

In step 220, an environmental complexity factor for the environment isdetermined based on the sensor data. In embodiments where a plurality ofsensors are used to collect environmental information, the sensor datacan be combined using suitable sensor fusion methods (e.g., Kalmanfilter, extended Kalman filter, unscented Kalman filter, or combinationsthereof). The combined or fused sensor data can then be used to generatea representation of the environment surrounding the UAV, including anyobstacles present in the environment. An environmental complexity factorcan then be computed based on the generated environmentalrepresentation. As previously described herein, the environmentalcomplexity factor can be used to represent the extent to which anenvironment is occupied by obstacles. The environmental complexityfactor may be a quantitative or qualitative measure. In someembodiments, the environmental complexity factor is determined based onone or more of: the number of obstacles, the volume or percentage ofspace occupied by obstacles, the volume or percentage of space within acertain proximity to the UAV occupied by obstacles, the volume orpercentage of space unobstructed by obstacles, the volume or percentageof space within a certain proximity to the UAV unobstructed byobstacles, the proximity of obstacles to the UAV, the obstacle density(e.g., number of obstacles per unit space), the types of obstacles(e.g., stationary or mobile), the spatial disposition of obstacles(e.g., position, orientation), the motion of obstacles (e.g., velocity,acceleration), and so on. For instance, an environment having arelatively high obstacle density would be associated with a highenvironmental complexity factor (e.g., indoor environment, urbanenvironment), whereas an environment having a relatively low obstacledensity would be associated with a low environmental complexity factor(e.g., high altitude environment). As another example, an environment inwhich a large percentage of space is occupied by obstacles would have ahigher complexity, whereas an environment having a large percentage ofunobstructed space would have a lower complexity. Exemplary methods fordetermining the environmental complexity factor of an environment areprovided in greater detail below.

In step 230, one or more operating rules for the operation of theunmanned aerial vehicle are determined based on the environmentalcomplexity factor. As previously described, at least some of theoperating rules can be configured to reduce the probability ofcollisions with environmental obstacles. Accordingly, the operatingrules for relatively complex environments may differ from the operatingrules for less complex environments, on the basis of the differingobstacle densities associated with such environments.

For example, the operating rules can include velocity rules configuredto minimize and/or prevent collisions between the UAV and obstacleswithin the environment. For instance, the velocity rules may provideconstraints for the velocity of the UAV (e.g., with respect to up tothree degrees of freedom in translation and up to three degrees offreedom in rotation) in order to ensure that the UAV is able to come toa stop (e.g., using automated anti-collision mechanisms or through userintervention) before colliding with an obstacle even when moving at themaximum allowable velocity. Accordingly, the UAV may be constrained tomove at lower velocities when within complex environments compared toless complex environments. In some embodiments, when operating within ahighly complex environment, the UAV may be constrained to move at avelocity of less than or equal to approximately 6 m/s, or within a rangeof 1 m/s to 6 m/s. Conversely, when operating within a less complexenvironment, the UAV may be constrained to move at a velocity of lessthan or equal to approximately 13 m/s, or within a range of 10 m/s to 15m/s. Optionally, the velocity rules may differ depending on the currentstage of operation of the UAV, including take off, landing, and duringflight. For instance, when the UAV is initially taking off and there isrelatively little sensor data available, the velocity rules may set alowered velocity limit for the UAV, such as approximately 7 m/s, orwithin a range from 5 m/s to 10 m/s. In some instances, it may bedesirable for the UAV to maintain a distance from environmentalobstacles that is greater than or equal to the minimum braking distance.Accordingly, the velocity rules may be determined based on the minimumbraking distance for the UAV. The minimum braking distance may vary withvelocity according to the relationv²=2aswhere v represents the velocity, a represents the maximum acceleration(e.g., 6.8 m/s²), and s represents the minimum braking distance to bringthe UAV to a stop (hover in place). For instance, when the UAV is movingwith a translational velocity of 7 m/s, the corresponding minimumbraking distance may be approximately 3 m to 4 m and it may takeapproximately 2 s to 3 s for the UAV to come to a stop.

Optionally, the operating rules can also be determined based on one ormore characteristics of the UAV or components thereof, such as the poweroutput of the propulsion system, braking capabilities, sensorreliability and accuracy, dimensions (e.g., length, width, height,volume), weight, and so on. For instance, a larger, heavier UAV maypresent more of a safety hazard than a smaller, lighter UAV, and maytherefore be subject to more stringent operating rules to preventcollisions, crash landings, and other accidents. Furthermore, theoperating rules can also be determined based on considerations of thequality of the user experience in piloting the UAV. The operating rulesmay be configured to avoid placing unnecessary or excessive constraintson manual control of the UAV. For instance, it may be desirable to allowthe user to operate the UAV at the maximum velocity limit and/or rangethat also fulfills the safety criteria described herein.

In some embodiments, the appropriate operating rules for a particularenvironment can be determined based on previously obtained flight data.The flight data can be collected during previous flights of the UAV orof other UAVs (e.g., of the same or similar model or type, or ofdifferent models or types) and can include information regarding UAVstate (e.g., position, orientation, velocity, acceleration, attitude,altitude, sensor data, sensor noise, or suitable combinations thereof),as well as environmental information and/or obstacle information aspreviously described herein. The flight data can be analyzed usingmachine learning techniques in order to provide analysis results thatinform the determination of the operating rules. For example, a machinelearning algorithm or model can be trained on the previously obtainedflight data in order to identify optimal operating rules for varioustypes of environmental conditions and contexts. Alternatively or incombination, big data processing techniques can be used to analyze thecollected flight data. The data analysis and/or machine learning can beperformed prior to operation of the UAV using an appropriate computingsystem, and the results can be made accessible to the UAV processor(s)(e.g., stored onboard the UAV) for use in determining the operatingrules.

In step 240, the UAV is caused to operate while complying with the oneor more operating rules. The operating rules may apply when the UAV isbeing controlled manually, semi-autonomously, or fully autonomously. Forinstance, the UAV may receive a signal (e.g., an input command from auser, a control instruction from an automated control system) indicatinga desired movement for the UAV (e.g., a desired translation, rotation,velocity, and/or acceleration). One or more onboard processors of theUAV (or any other suitable system or device provided herein) candetermine whether the UAV can move as instructed by the received signalwhile complying with the operating rules. If so, the processors cangenerate control instructions that are transmitted to the UAV propulsionsystem in order to cause the UAV to execute the movement. If not, theprocessors can modify the signal so that the resultant movement is incompliance with the operating rules and provide the correspondinginstructions to the propulsion system. For example, if a user instructsthe UAV to move at a velocity exceeding a determined velocity limit, theprocessors can instead cause the UAV to move at a velocity less than orequal to the maximum velocity specified by the velocity limit. In suchsituations, the UAV may not exceed the maximum velocity regardless ofthe input provided by the user. Furthermore, one or more of the sensorspreviously described herein can be used to measure the current state(e.g., position, orientation, velocity, acceleration) of the UAV andprovide feedback to the UAV control system to ensure that the operatingrules are continuously obeyed. Various techniques can be applied toreduce the noise of the sensor measurements so as to improve thestability of UAV control, such as filtering, sensor fusion (e.g., usingKalman filters), time-averaging, and so on.

Although the above steps show method 200 of controlling a UAV inaccordance with embodiments, a person of ordinary skill in the art willrecognize many variations based on the teachings described herein. Someof the steps may comprise sub-steps. In some embodiments, step 220 isoptional, such that in step 230 the operating rules are determined basedon the sensor data without considering the environmental complexityfactor. Many of the steps may be repeated as often as is beneficial. Forexample, steps 210, 220, 230, and 240 can be repeated continuously or atpredetermined time intervals during the operation of the UAV so as toprovide dynamic and adaptive determination of the operating rules.Accordingly, the method 200 can be used to detect a change in theobstacle content (e.g., obstacle density) of the environment, asindicated by a change in the determined environmental complexity factor.The operating rules can subsequently be modified to reflect the changein the environmental complexity factor, thereby dynamically adapting theUAV control scheme based on the current environmental context. Forinstance, if a decrease in obstacle density is detected (as evidenced bya decrease in the environmental complexity factor), the velocity limitmay be increased. Conversely, if an increase in obstacle density isdetected (corresponding to an increased in environmental complexity),the velocity limit may be decreased.

FIG. 3 illustrates a method 300 for controlling a UAV, in accordancewith embodiments. The steps of the method 300 can be combined with orsubstituted for any of the steps of the other methods provided herein.Additionally, any step of the method 300 can be repeated, omitted, orcombined with other steps as desired. Similar to the method 200, one ormore steps of the method 300 can be performed in real-time duringoperation of the UAV.

In step 310, sensor data of the environment is received from one or moresensors. Similar to step 210 of the method 200, the one or more sensorsmay be sensors of different types. For example, the UAV may carry one ormore vision sensors (e.g., a stereovision sensor including a pair ofcameras) and one or more proximity sensors (e.g., a lidar sensor,ultrasonic sensor, time-of-flight camera). The vision sensors andproximity sensors can be configured to capture data of the environmentsurrounding the UAV, such as data regarding the geometry, density, andspatial disposition of obstacles within the environment. In someembodiments, the sensors may span a 360° field of view around the UAV,and may provide information regarding obstacles situated within a 40 mradius from the UAV.

In step 320, the sensor data is pre-processed. The pre-processing may beperformed to correct sensor errors, improve the signal-to-noise ratioand/or to enhance the quality and reliability of the sensor data. Anysuitable data pre-processing technique can be used, such as filtering,time-averaging, and the like. For instance, for image data, imageprocessing methods such as histogram equalization can be used to improveimage data quality. Optionally, data from redundant sensors can becombined in order to reduce the effects of noise or errors from any onesensor.

In step 330, a three-dimensional (3D) digital reconstruction of theenvironment is generated based on the pre-processed sensor data. Inembodiments where multiple sensors are used to capture environmentalinformation, the 3D reconstruction can be produced by combining the datafrom each sensor using suitable sensor fusion methods (e.g., Kalmanfilters, extended Kalman filters, unscented Kalman filters, etc.). Thesensor data may already be provided in a 3D format or otherwise include3D environmental information. For example, proximity sensors such aslidar sensors or ultrasonic sensors may generate environmental data inthe form of 3D point clouds. Alternatively, other types of sensor datamay require additional processing in order to produce a 3Drepresentation. For example, a stereovision sensor may include a pair ofcameras, each being used to obtain a respective set of image data. Theimage data captured by each camera can be analyzed using appropriatecomputer vision algorithms in order to generate a 3D model representingthe imaged environment. In some embodiments, the image data from bothcameras can be compared in order to determine the disparity between eachpair of images and thereby generate a disparity map that includes depthinformation for environmental obstacles.

FIG. 4 illustrates an exemplary disparity map 400 generated from sensordata of an environment, in accordance with embodiments. In the disparitymap 400, light-colored regions represent objects that are relativelyclose to the cameras and dark-colored regions represent objects that arerelatively far from the cameras. The disparity map may be generatedbased on image data from a pair of cameras as described above. Forexample, the 3D coordinates (x_(w), y_(w), z_(w)) of a point P on thesurface of an object in the environment can be determined by solving

$x_{w} = {\left( {u - u_{0}} \right) \times \frac{b}{d}}$$y_{w} = {\left( {v - v_{0}} \right) \times \frac{b}{d}}$$z_{w} = {f \times \frac{b}{d}}$where u, v represent the coordinates of P in the image data, u₀, v₀represent the internal parameters of the cameras, f is the focal length,b is the distance between cameras, and d is the disparity between thepaired image data.

Alternatively or in combination, other types of 3D digitalreconstructions can also be used. For instance, an occupancy grid can beused to represent the spatial disposition of obstacles within theenvironment. The environment can be represented as a three-dimensionalgrid, with each location either being occupied by an obstacle,unoccupied, or unknown (e.g., due to insufficient or unreliable sensordata). In some embodiments, the occupancy grid can be generated based bycombining data from vision sensors and proximity sensors. The occupancygrid can then be analyzed to determine various metrics (e.g., totaloccupied volume, total unoccupied volume, ratio of occupied tounoccupied space) that can be used to determine the environmentalcomplexity factor.

In step 340, an environmental complexity factor is determined based onthe 3D reconstruction. Any suitable technique can be used in order todetermine the environmental complexity factor. For example, in someembodiments, the 3D reconstruction is used to calculate variousobstacle-related statistics, such as the number of obstacles at eachlocation within the environment. The calculated statistics can then beanalyzed in order to determine the overall entropy of the environment,which may be related to the environmental complexity (e.g., a highentropy corresponds to a high environmental complexity factor, a lowentropy corresponds to a low environmental complexity factor). Variousapproaches can be used to identify obstacles in the 3D environmentalreconstruction and/or the sensor data, such as computer vision and imagerecognition methods. For example, a feature extraction algorithm can beimplemented in order to detect features (e.g., histogram of orientedgradients (HOG) features in image data, surface normal features in 3Dpoint cloud data) indicative of an obstacle. Classification algorithmscan be used to distinguish between obstacles and non-obstacles in theenvironmental reconstruction. In some embodiments, the environmentalcomplexity analysis techniques described herein may utilize suitablytrained machine learning algorithms and models to perform featureextraction and/or obstacle detection and classification.

FIG. 5 illustrates a histogram 500 indicative of obstacle density in anenvironment, in accordance with embodiments. The histogram 500 can begenerated based on a 3D environmental reconstruction, such as a 3D pointcloud, 3D disparity map (e.g., the map 400 of FIG. 4), or a 3D occupancygrid, by determining the number of obstacles at each location in thereconstruction. In the depiction of FIG. 5, the histogram 500 provides agraphical representation of the obstacle frequency versus the distancefrom the UAV. The histogram distribution may vary based on the spatialdisposition and distribution of obstacles within the environment. Forinstance, in situations where there is a relatively high obstacledensity near the UAV, the obstacle frequency would be higher for theclose distance histogram bins versus the long distance bins. Conversely,when the majority of obstacles are relatively far from the UAV, theobstacle frequency would be lower for the close distance bins comparedto the long distance bins. Furthermore, in environments with lowobstacle densities, the histogram 500 may have a relatively uniformdistribution (e.g., low entropy), whereas high obstacle densities mayresult in the histogram 500 having a non-uniform distribution (e.g.,high entropy). The characteristics and statistics of the histogram 500(e.g., the distribution pattern, average, median, mode, entropy) may beused to determine the environmental complexity factor.

In step 350, one or more operating rules are determined based on theenvironmental complexity factor. The operating rules can be selected inorder to provide appropriate limits, ranges, and/or values for variousUAV operating parameters for an environment having a given complexityfactor in order to minimize the potential for collisions with obstacles,as previously described herein. Additionally, suitable noise reductionmethods can be applied during determination of the operating rules inorder to avoid undesirable fluctuations in the control scheme caused bysensor noise or error. In some instances, the operating rules determinedat sequential time points may vary substantially, e.g., due to noisysensor data or limitations in sensor precision. To counteract thisinstability, the operating rules determined during a current time pointcan be compared to one or more operating rules determined during one ormore previous time points. The current and previously determinedoperating rules can be combined (e.g., averaged) in order to generate afinal set of operating rules that are applied to control the operationof the UAV. Alternatively or in combination, suitable filtering methodscan be applied to reduce the “noisiness” of operating ruledetermination.

FIG. 6 illustrates a first-in-first-out (FIFO) queue 600 for determiningvelocity rules, in accordance with embodiments. The FIFO queue 600 canbe used to combine velocity rules determined at various time pointsduring operation of the UAV in order to improve the stability of the UAVvelocity control scheme. Although the FIFO queue 600 is presented in thecontext of velocity rules, the queue 600 can be applied to thedetermination of other types of operating rules, such as the embodimentspreviously described herein.

The queue 600 includes a plurality of previously determined velocityrules 602 a-f, each determined at a corresponding time point prior tothe current time point t. The queue 600 can store any number ofpreviously determined velocity rules 602 a-f, such as one, two, three,four, five, six, or more previously determined velocity rules. Thepreviously determined velocity rules 602 a-f can each be generated usingany of the methods described herein and added sequentially to the queue600. For instance, velocity rules 602 b may have been added to the queue600 at time t-2. Subsequently, velocity rules 602 a may have been addedto the queue 600 at point t-1. At the current time point t, the mostcurrent velocity rules 604 can be determined as described herein at thecurrent time t and added to the queue 600. As the most current velocityrules 604 are added, the oldest set of velocity rules 602 f can beremoved from the queue 600. This process can be repeated duringoperation of the UAV such that the queue 600 is continuously updated tostore the most recently determined velocity rules. The interval betweeneach time point can be constant or may vary. For instance, the queue 600can be updated approximately every 0.1 s to add newly determinedvelocity rules and remove the oldest velocity rules.

The velocity rules stored in the queue 600 can be combined in anysuitable manner in order to determine the final velocity rules forcontrolling the UAV. In some embodiments, the final velocity rules aredetermined from a weighted average of the stored velocity rules. Theweighted average may assign a greater weight to more recently determinedvelocity rules and a smaller weight to older velocity rules. Forexample, the following equation may be applied to determine a finalvelocity limit V_(f):V _(f)=0.1V _(t-4)+0.15V _(t-3)+0.2V _(t-2)+0.25V _(t-1)+0.3V _(t)where V_(t-4), V_(t-3), V_(t-2), V_(t-1), and V_(t) are the velocitylimits determined at times t-4, t-3, t-2, t-1, and t, respectively.

In step 360, additional operating rules for the UAV are determined. Theadditional operating rules may be determined independently of theenvironmental complexity factor, and may therefore differ from the rulesdetermined in step 350. The additional operating rules can include anyembodiment of the operating rules presented herein. In some embodiments,the additional rules can be configured to ensure smooth and stable UAVflight. As previously mentioned, the UAV attitude may be related to UAVacceleration, such that as the UAV assumes a certain attitude whenaccelerating or decelerating to a specified velocity. However, extremeattitudes and/or sudden attitude changes may be undesirable for flightstability, power consumption, and certain UAV applications (e.g., aerialphotography using a camera carried by the UAV). Rather, it may bepreferable for the UAV to achieve a desired velocity by incremental orgradual acceleration resulting in more moderate UAV attitudes.Accordingly, suitable attitude rules such as an attitude limit and/orrange for UAV operation can be determined in order to ensure smooth andstable UAV movements. The attitude rules can be varied as necessary inorder to optimize the UAV attitude control scheme. An attitude limit maybe less than or equal to approximately 35° relative to the vertical axisof the UAV. An attitude range may be within a range of 25° to 45° fromthe vertical axis. Similarly, suitable acceleration rules such as anacceleration limit and/or range can be determined to provide smooth UAVflight, which may be related to a corresponding attitude limit and/orrange. Alternatively or in combination, appropriate UAV attitude (oracceleration) values can be determined by using low pass filters and/orother smoothing algorithms to reduce or eliminate sudden attitudechanges (or acceleration changes). As another example, the systems anddevices described herein can determine appropriate altitude rules suchas an altitude range for the UAV. A UAV operating at high altitudes maybe permitted to move at higher velocities than a UAV operating at lowaltitudes.

In step 370, the UAV is caused to operate (e.g., by one or moreprocessors, or any other systems or devices described herein) whilecomplying with the one or more operating rules and the other controlrules. Similar to step 240 of the method 200, the UAV may receivesignals indicating a desired movement, and may adjust the desiredmovement as necessary to ensure that the operating rules and/or controlrules are observed. Furthermore, real-time sensor measurements can beused to monitor the UAV state and provide feedback to ensure that theUAV continues to obey the operating rules and/or other control rules.

Although some embodiments herein may present approaches for fullyautomated context-based determination of UAV operating rules, otherembodiments may provide mechanisms for receiving user input indicativeof user preferences for the UAV control scheme. For example, a useroperating the UAV may, based on their own judgment regarding the currentflight environment and obstacle density, determine a set of desiredoperating rules for operating the UAV. Accordingly, embodiments hereinmay utilize user input in addition to environmental considerations suchas the environmental complexity factor when determining an appropriateset of operating rules.

FIG. 7 illustrates a method 700 for controlling a UAV, in accordancewith embodiments. The steps of the method 700 can be combined with orsubstituted for any of the steps of the other methods provided herein.Additionally, any step of the method 700 can be repeated, omitted, orcombined with other steps as desired. One or more steps of the method700 can be performed in real-time during operation of the UAV.

In step 710, a first set of operating rules for the UAV are determinedbased on sensor data, using any of the techniques presented herein.

In step 720, user input indicating a second set of operating rules forthe UAV is received. The user may provide input data that defines orselects the second set of operating rules for the UAV. Optionally,rather than directly inputting the operating rules, the user mayindicate a preferred flight mode that is associated with a set ofoperating rules, as previously described herein. For instance, a usermay be able to select a low velocity flight mode, an intermediatevelocity flight mode, or a high velocity flight mode. Similarly, indoorand outdoor flight modes may also be associated with distinct operatingrules. The user inputs for selecting the operating rules and/or flightmode may be provided to the UAV via a suitable remote controller (alsoreferred to herein as a “terminal”), mobile device, or other inputdevice in communication with the UAV. Accordingly, when desired, theuser can transmit commands to the UAV indicating a preferred controlscheme.

FIG. 8 illustrates a remote controller 800 for controlling a UAV, inaccordance with embodiments. The remote controller 800 can include acontroller body 802 sized to held by a user. One or more joysticks 804can be mounted on the body 802 and used as input mechanisms forcontrolling the spatial disposition of the UAV. The body 802 can alsoinclude other types of input mechanisms, such as one or more switches806, a knob 808, and/or a slide switch 810. Optionally, alternativeinput mechanisms such as buttons, dials, touchscreens, keypads, voicecontrols, gesture controls, inertial sensors to detect the tilt and/orattitude of the controller 800, etc. can also be used. The inputmechanisms can be manually set by a user to one or more positions, eachof the positions corresponding to a predetermined input for controllingthe UAV. For example, the knob 808 can be turned (e.g., along acounterclockwise direction as indicated by the arrows) to a plurality ofstops each representing a different respective input value.

In some embodiments, the input mechanisms can be manipulated by a userto input user-preferred operating rules and/or flight modes. Forinstance, each stop of the knob 808 may correspond to a flight modeassociated with a set of operating rules. The user can turn the knob 808to the appropriate stop in order to indicate the desired flight mode foroperating the UAV. Similarly, the user may use other input mechanismssuch as the switches 806 and/or 810 in order to select a flight mode.Additionally or in combination, the user input can be provided usingother types of devices besides the remote controller 800. For example,the UAV may be in communication with a mobile device (e.g., smartphone)running suitable mobile application software (“app”) that can generatecontrol instructions for the UAV. The app may be configured to obtaininputs from a user indicating the desired operating rules. The user maydirectly input the operating rules, or may select a flight modeassociated with a set of operating rules, as described above.

In step 730, one of the first or second sets of operating rules isselected to be used to control the UAV. In situations where theenvironmental complexity factor suggests a first set of operating rulesand the user input indicates a second, different set of operating rules,suitable approaches can be applied to select one of the two sets for usein controlling the UAV. For example, some embodiments herein maypreferentially select the “safest” set (e.g., the set that is leastlikely to result in collisions with obstacles). If the user specifies afirst velocity limit μ_(A) and the determined environmental complexityfactor corresponds to a second velocity limit μ_(B), the actual velocitylimit μ to be used may be determined by selecting the smaller of the twovelocity limits (e.g., μ=min(μ_(A),μ_(B))).

In step 740, the UAV is caused to be operated while complying with theselected one of the first or second sets of operating rules, aspreviously described herein.

The systems, devices, and methods described herein can be applied to awide variety of movable objects. As previously mentioned, anydescription herein of an aerial vehicle may apply to and be used for anymovable object. A movable object of the present invention can beconfigured to move within any suitable environment, such as in air(e.g., a fixed-wing aircraft, a rotary-wing aircraft, or an aircrafthaving neither fixed wings nor rotary wings), in water (e.g., a ship ora submarine), on ground (e.g., a motor vehicle, such as a car, truck,bus, van, motorcycle; a movable structure or frame such as a stick,fishing pole; or a train), under the ground (e.g., a subway), in space(e.g., a spaceplane, a satellite, or a probe), or any combination ofthese environments. The movable object can be a vehicle, such as avehicle described elsewhere herein. In some embodiments, the movableobject can be mounted on a living subject, such as a human or an animal.Suitable animals can include avines, canines, felines, equines, bovines,ovines, porcines, delphines, rodents, or insects.

The movable object may be capable of moving freely within theenvironment with respect to six degrees of freedom (e.g., three degreesof freedom in translation and three degrees of freedom in rotation).Alternatively, the movement of the movable object can be constrainedwith respect to one or more degrees of freedom, such as by apredetermined path, track, or orientation. The movement can be actuatedby any suitable actuation mechanism, such as an engine or a motor. Theactuation mechanism of the movable object can be powered by any suitableenergy source, such as electrical energy, magnetic energy, solar energy,wind energy, gravitational energy, chemical energy, nuclear energy, orany suitable combination thereof. The movable object may beself-propelled via a propulsion system, as described elsewhere herein.The propulsion system may optionally run on an energy source, such aselectrical energy, magnetic energy, solar energy, wind energy,gravitational energy, chemical energy, nuclear energy, or any suitablecombination thereof. Alternatively, the movable object may be carried bya living being.

In some instances, the movable object can be a vehicle. Suitablevehicles may include water vehicles, aerial vehicles, space vehicles, orground vehicles. For example, aerial vehicles may be fixed-wing aircraft(e.g., airplane, gliders), rotary-wing aircraft (e.g., helicopters,rotorcraft), aircraft having both fixed wings and rotary wings, oraircraft having neither (e.g., blimps, hot air balloons). A vehicle canbe self-propelled, such as self-propelled through the air, on or inwater, in space, or on or under the ground. A self-propelled vehicle canutilize a propulsion system, such as a propulsion system including oneor more engines, motors, wheels, axles, magnets, rotors, propellers,blades, nozzles, or any suitable combination thereof. In some instances,the propulsion system can be used to enable the movable object to takeoff from a surface, land on a surface, maintain its current positionand/or orientation (e.g., hover), change orientation, and/or changeposition.

The movable object can be controlled remotely by a user or controlledlocally by an occupant within or on the movable object. In someembodiments, the movable object is an unmanned movable object, such as aUAV. An unmanned movable object, such as a UAV, may not have an occupantonboard the movable object. The movable object can be controlled by ahuman or an autonomous control system (e.g., a computer control system),or any suitable combination thereof. The movable object can be anautonomous or semi-autonomous robot, such as a robot configured with anartificial intelligence.

The movable object can have any suitable size and/or dimensions. In someembodiments, the movable object may be of a size and/or dimensions tohave a human occupant within or on the vehicle. Alternatively, themovable object may be of size and/or dimensions smaller than thatcapable of having a human occupant within or on the vehicle. The movableobject may be of a size and/or dimensions suitable for being lifted orcarried by a human. Alternatively, the movable object may be larger thana size and/or dimensions suitable for being lifted or carried by ahuman. In some instances, the movable object may have a maximumdimension (e.g., length, width, height, diameter, diagonal) of less thanor equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. Themaximum dimension may be greater than or equal to about: 2 cm, 5 cm, 10cm, 50 cm, 1 m, 2 m, 5 m, or 10 m. For example, the distance betweenshafts of opposite rotors of the movable object may be less than orequal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m, or 10 m.Alternatively, the distance between shafts of opposite rotors may begreater than or equal to about: 2 cm, 5 cm, 10 cm, 50 cm, 1 m, 2 m, 5 m,or 10 m.

In some embodiments, the movable object may have a volume of less than100 cm×100 cm×100 cm, less than 50 cm×50 cm×30 cm, or less than 5 cm×5cm×3 cm. The total volume of the movable object may be less than orequal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40 cm³, 50cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³, 300 cm³,500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³, 1 m³, or10 m³. Conversely, the total volume of the movable object may be greaterthan or equal to about: 1 cm³, 2 cm³, 5 cm³, 10 cm³, 20 cm³, 30 cm³, 40cm³, 50 cm³, 60 cm³, 70 cm³, 80 cm³, 90 cm³, 100 cm³, 150 cm³, 200 cm³,300 cm³, 500 cm³, 750 cm³, 1000 cm³, 5000 cm³, 10,000 cm³, 100,000 cm³,1 m³, or 10 m³.

In some embodiments, the movable object may have a footprint (which mayrefer to the lateral cross-sectional area encompassed by the movableobject) less than or equal to about: 32,000 cm², 20,000 cm², 10,000 cm²,1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm². Conversely, thefootprint may be greater than or equal to about: 32,000 cm², 20,000 cm²,10,000 cm², 1,000 cm², 500 cm², 100 cm², 50 cm², 10 cm², or 5 cm².

In some instances, the movable object may weigh no more than 1000 kg.The weight of the movable object may be less than or equal to about:1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60 kg, 50kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10 kg, 9 kg,8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1 kg, 0.05 kg,or 0.01 kg. Conversely, the weight may be greater than or equal toabout: 1000 kg, 750 kg, 500 kg, 200 kg, 150 kg, 100 kg, 80 kg, 70 kg, 60kg, 50 kg, 45 kg, 40 kg, 35 kg, 30 kg, 25 kg, 20 kg, 15 kg, 12 kg, 10kg, 9 kg, 8 kg, 7 kg, 6 kg, 5 kg, 4 kg, 3 kg, 2 kg, 1 kg, 0.5 kg, 0.1kg, 0.05 kg, or 0.01 kg.

In some embodiments, a movable object may be small relative to a loadcarried by the movable object. The load may include a payload and/or acarrier, as described in further detail below. In some examples, a ratioof a movable object weight to a load weight may be greater than, lessthan, or equal to about 1:1. In some instances, a ratio of a movableobject weight to a load weight may be greater than, less than, or equalto about 1:1. Optionally, a ratio of a carrier weight to a load weightmay be greater than, less than, or equal to about 1:1. When desired, theratio of an movable object weight to a load weight may be less than orequal to: 1:2, 1:3, 1:4, 1:5, 1:10, or even less. Conversely, the ratioof a movable object weight to a load weight can also be greater than orequal to: 2:1, 3:1, 4:1, 5:1, 10:1, or even greater.

In some embodiments, the movable object may have low energy consumption.For example, the movable object may use less than about: 5 W/h, 4 W/h, 3W/h, 2 W/h, 1 W/h, or less. In some instances, a carrier of the movableobject may have low energy consumption. For example, the carrier may useless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less. Optionally,a payload of the movable object may have low energy consumption, such asless than about: 5 W/h, 4 W/h, 3 W/h, 2 W/h, 1 W/h, or less.

FIG. 9 illustrates a UAV 900, in accordance with embodiments of thepresent invention. The UAV may be an example of a movable object asdescribed herein. The UAV 900 can include a propulsion system havingfour rotors 902, 904, 906, and 908. Any number of rotors may be provided(e.g., one, two, three, four, five, six, or more). The rotors can beembodiments of the self-tightening rotors described elsewhere herein.The rotors, rotor assemblies, or other propulsion systems of theunmanned aerial vehicle may enable the unmanned aerial vehicle tohover/maintain position, change orientation, and/or change location. Thedistance between shafts of opposite rotors can be any suitable length910. For example, the length 910 can be less than or equal to 2 m, orless than equal to 5 m. In some embodiments, the length 910 can bewithin a range from 40 cm to 1 m, from 10 cm to 2 m, or from 5 cm to 5m. Any description herein of a UAV may apply to a movable object, suchas a movable object of a different type, and vice versa.

In some embodiments, the movable object can be configured to carry aload. The load can include one or more of passengers, cargo, equipment,instruments, and the like. The load can be provided within a housing.The housing may be separate from a housing of the movable object, or bepart of a housing for an movable object. Alternatively, the load can beprovided with a housing while the movable object does not have ahousing. Alternatively, portions of the load or the entire load can beprovided without a housing. The load can be rigidly fixed relative tothe movable object. Optionally, the load can be movable relative to themovable object (e.g., translatable or rotatable relative to the movableobject).

In some embodiments, the load includes a payload. The payload can beconfigured not to perform any operation or function. Alternatively, thepayload can be a payload configured to perform an operation or function,also known as a functional payload. For example, the payload can includeone or more sensors for surveying one or more targets. Any suitablesensor can be incorporated into the payload, such as an image capturedevice (e.g., a camera), an audio capture device (e.g., a parabolicmicrophone), an infrared imaging device, or an ultraviolet imagingdevice. The sensor can provide static sensing data (e.g., a photograph)or dynamic sensing data (e.g., a video). In some embodiments, the sensorprovides sensing data for the target of the payload. Alternatively or incombination, the payload can include one or more emitters for providingsignals to one or more targets. Any suitable emitter can be used, suchas an illumination source or a sound source. In some embodiments, thepayload includes one or more transceivers, such as for communicationwith a module remote from the movable object. Optionally, the payloadcan be configured to interact with the environment or a target. Forexample, the payload can include a tool, instrument, or mechanismcapable of manipulating objects, such as a robotic arm.

Optionally, the load may include a carrier. The carrier can be providedfor the payload and the payload can be coupled to the movable object viathe carrier, either directly (e.g., directly contacting the movableobject) or indirectly (e.g., not contacting the movable object).Conversely, the payload can be mounted on the movable object withoutrequiring a carrier. The payload can be integrally formed with thecarrier. Alternatively, the payload can be releasably coupled to thecarrier. In some embodiments, the payload can include one or morepayload elements, and one or more of the payload elements can be movablerelative to the movable object and/or the carrier, as described above.

The carrier can be integrally formed with the movable object.Alternatively, the carrier can be releasably coupled to the movableobject. The carrier can be coupled to the movable object directly orindirectly. The carrier can provide support to the payload (e.g., carryat least part of the weight of the payload). The carrier can include asuitable mounting structure (e.g., a gimbal platform) capable ofstabilizing and/or directing the movement of the payload. In someembodiments, the carrier can be adapted to control the state of thepayload (e.g., position and/or orientation) relative to the movableobject. For example, the carrier can be configured to move relative tothe movable object (e.g., with respect to one, two, or three degrees oftranslation and/or one, two, or three degrees of rotation) such that thepayload maintains its position and/or orientation relative to a suitablereference frame regardless of the movement of the movable object. Thereference frame can be a fixed reference frame (e.g., the surroundingenvironment). Alternatively, the reference frame can be a movingreference frame (e.g., the movable object, a payload target).

In some embodiments, the carrier can be configured to permit movement ofthe payload relative to the carrier and/or movable object. The movementcan be a translation with respect to up to three degrees of freedom(e.g., along one, two, or three axes) or a rotation with respect to upto three degrees of freedom (e.g., about one, two, or three axes), orany suitable combination thereof.

In some instances, the carrier can include a carrier frame assembly anda carrier actuation assembly. The carrier frame assembly can providestructural support to the payload. The carrier frame assembly caninclude individual carrier frame components, some of which can bemovable relative to one another. The carrier actuation assembly caninclude one or more actuators (e.g., motors) that actuate movement ofthe individual carrier frame components. The actuators can permit themovement of multiple carrier frame components simultaneously, or may beconfigured to permit the movement of a single carrier frame component ata time. The movement of the carrier frame components can produce acorresponding movement of the payload. For example, the carrieractuation assembly can actuate a rotation of one or more carrier framecomponents about one or more axes of rotation (e.g., roll axis, pitchaxis, or yaw axis). The rotation of the one or more carrier framecomponents can cause a payload to rotate about one or more axes ofrotation relative to the movable object. Alternatively or incombination, the carrier actuation assembly can actuate a translation ofone or more carrier frame components along one or more axes oftranslation, and thereby produce a translation of the payload along oneor more corresponding axes relative to the movable object.

In some embodiments, the movement of the movable object, carrier, andpayload relative to a fixed reference frame (e.g., the surroundingenvironment) and/or to each other, can be controlled by a terminal. Theterminal can be a remote control device at a location distant from themovable object, carrier, and/or payload. The terminal can be disposed onor affixed to a support platform. Alternatively, the terminal can be ahandheld or wearable device. For example, the terminal can include asmartphone, tablet, laptop, computer, glasses, gloves, helmet,microphone, or suitable combinations thereof. The terminal can include auser interface, such as a keyboard, mouse, joystick, touchscreen, ordisplay. Any suitable user input can be used to interact with theterminal, such as manually entered commands, voice control, gesturecontrol, or position control (e.g., via a movement, location or tilt ofthe terminal).

The terminal can be used to control any suitable state of the movableobject, carrier, and/or payload. For example, the terminal can be usedto control the position and/or orientation of the movable object,carrier, and/or payload relative to a fixed reference from and/or toeach other. In some embodiments, the terminal can be used to controlindividual elements of the movable object, carrier, and/or payload, suchas the actuation assembly of the carrier, a sensor of the payload, or anemitter of the payload. The terminal can include a wirelesscommunication device adapted to communicate with one or more of themovable object, carrier, or payload.

The terminal can include a suitable display unit for viewing informationof the movable object, carrier, and/or payload. For example, theterminal can be configured to display information of the movable object,carrier, and/or payload with respect to position, translationalvelocity, translational acceleration, orientation, angular velocity,angular acceleration, or any suitable combinations thereof. In someembodiments, the terminal can display information provided by thepayload, such as data provided by a functional payload (e.g., imagesrecorded by a camera or other image capturing device).

Optionally, the same terminal may both control the movable object,carrier, and/or payload, or a state of the movable object, carrierand/or payload, as well as receive and/or display information from themovable object, carrier and/or payload. For example, a terminal maycontrol the positioning of the payload relative to an environment, whiledisplaying image data captured by the payload, or information about theposition of the payload. Alternatively, different terminals may be usedfor different functions. For example, a first terminal may controlmovement or a state of the movable object, carrier, and/or payload whilea second terminal may receive and/or display information from themovable object, carrier, and/or payload. For example, a first terminalmay be used to control the positioning of the payload relative to anenvironment while a second terminal displays image data captured by thepayload. Various communication modes may be utilized between a movableobject and an integrated terminal that both controls the movable objectand receives data, or between the movable object and multiple terminalsthat both control the movable object and receives data. For example, atleast two different communication modes may be formed between themovable object and the terminal that both controls the movable objectand receives data from the movable object.

FIG. 10 illustrates a movable object 1000 including a carrier 1002 and apayload 1004, in accordance with embodiments. Although the movableobject 1000 is depicted as an aircraft, this depiction is not intendedto be limiting, and any suitable type of movable object can be used, aspreviously described herein. One of skill in the art would appreciatethat any of the embodiments described herein in the context of aircraftsystems can be applied to any suitable movable object (e.g., a UAV). Insome instances, the payload 1004 may be provided on the movable object1000 without requiring the carrier 1002. The movable object 1000 mayinclude propulsion mechanisms 1006, a sensing system 1008, and acommunication system 1010.

The propulsion mechanisms 1006 can include one or more of rotors,propellers, blades, engines, motors, wheels, axles, magnets, or nozzles,as previously described. For example, the propulsion mechanisms 1006 maybe self-tightening rotors, rotor assemblies, or other rotary propulsionunits, as disclosed elsewhere herein. The movable object may have one ormore, two or more, three or more, or four or more propulsion mechanisms.The propulsion mechanisms may all be of the same type. Alternatively,one or more propulsion mechanisms can be different types of propulsionmechanisms. The propulsion mechanisms 1006 can be mounted on the movableobject 1000 using any suitable means, such as a support element (e.g., adrive shaft) as described elsewhere herein. The propulsion mechanisms1006 can be mounted on any suitable portion of the movable object 1000,such on the top, bottom, front, back, sides, or suitable combinationsthereof.

In some embodiments, the propulsion mechanisms 1006 can enable themovable object 1000 to take off vertically from a surface or landvertically on a surface without requiring any horizontal movement of themovable object 1000 (e.g., without traveling down a runway). Optionally,the propulsion mechanisms 1006 can be operable to permit the movableobject 1000 to hover in the air at a specified position and/ororientation. One or more of the propulsion mechanism 1000 may becontrolled independently of the other propulsion mechanisms.Alternatively, the propulsion mechanisms 1000 can be configured to becontrolled simultaneously. For example, the movable object 1000 can havemultiple horizontally oriented rotors that can provide lift and/orthrust to the movable object. The multiple horizontally oriented rotorscan be actuated to provide vertical takeoff, vertical landing, andhovering capabilities to the movable object 1000. In some embodiments,one or more of the horizontally oriented rotors may spin in a clockwisedirection, while one or more of the horizontally rotors may spin in acounterclockwise direction. For example, the number of clockwise rotorsmay be equal to the number of counterclockwise rotors. The rotation rateof each of the horizontally oriented rotors can be varied independentlyin order to control the lift and/or thrust produced by each rotor, andthereby adjust the spatial disposition, velocity, and/or acceleration ofthe movable object 1000 (e.g., with respect to up to three degrees oftranslation and up to three degrees of rotation).

The sensing system 1008 can include one or more sensors that may sensethe spatial disposition, velocity, and/or acceleration of the movableobject 1000 (e.g., with respect to up to three degrees of translationand up to three degrees of rotation). The one or more sensors caninclude any of the sensors previously described herein, including GPSsensors, motion sensors, inertial sensors, proximity sensors, or imagesensors. The sensing data provided by the sensing system 1008 can beused to control the spatial disposition, velocity, and/or orientation ofthe movable object 1000 (e.g., using a suitable processing unit and/orcontrol module, as described below). Alternatively, the sensing system1008 can be used to provide data regarding the environment surroundingthe movable object, such as weather conditions, proximity to potentialobstacles, location of geographical features, location of manmadestructures, and the like.

The communication system 1010 enables communication with terminal 1012having a communication system 1014 via wireless signals 1016. Thecommunication systems 1010, 1014 may include any number of transmitters,receivers, and/or transceivers suitable for wireless communication. Thecommunication may be one-way communication, such that data can betransmitted in only one direction. For example, one-way communicationmay involve only the movable object 1000 transmitting data to theterminal 1012, or vice-versa. The data may be transmitted from one ormore transmitters of the communication system 1010 to one or morereceivers of the communication system 1012, or vice-versa.Alternatively, the communication may be two-way communication, such thatdata can be transmitted in both directions between the movable object1000 and the terminal 1012. The two-way communication can involvetransmitting data from one or more transmitters of the communicationsystem 1010 to one or more receivers of the communication system 1014,and vice-versa.

In some embodiments, the terminal 1012 can provide control data to oneor more of the movable object 1000, carrier 1002, and payload 1004 andreceive information from one or more of the movable object 1000, carrier1002, and payload 1004 (e.g., position and/or motion information of themovable object, carrier or payload; data sensed by the payload such asimage data captured by a payload camera). In some instances, controldata from the terminal may include instructions for relative positions,movements, actuations, or controls of the movable object, carrier and/orpayload. For example, the control data may result in a modification ofthe location and/or orientation of the movable object (e.g., via controlof the propulsion mechanisms 1006), or a movement of the payload withrespect to the movable object (e.g., via control of the carrier 1002).The control data from the terminal may result in control of the payload,such as control of the operation of a camera or other image capturingdevice (e.g., taking still or moving pictures, zooming in or out,turning on or off, switching imaging modes, change image resolution,changing focus, changing depth of field, changing exposure time,changing viewing angle or field of view). In some instances, thecommunications from the movable object, carrier and/or payload mayinclude information from one or more sensors (e.g., of the sensingsystem 1008 or of the payload 1004). The communications may includesensed information from one or more different types of sensors (e.g.,GPS sensors, motion sensors, inertial sensor, proximity sensors, orimage sensors). Such information may pertain to the position (e.g.,location, orientation), movement, or acceleration of the movable object,carrier and/or payload. Such information from a payload may include datacaptured by the payload or a sensed state of the payload. The controldata provided transmitted by the terminal 1012 can be configured tocontrol a state of one or more of the movable object 1000, carrier 1002,or payload 1004. Alternatively or in combination, the carrier 1002 andpayload 1004 can also each include a communication module configured tocommunicate with terminal 1012, such that the terminal can communicatewith and control each of the movable object 1000, carrier 1002, andpayload 1004 independently.

In some embodiments, the movable object 1000 can be configured tocommunicate with another remote device in addition to the terminal 1012,or instead of the terminal 1012. The terminal 1012 may also beconfigured to communicate with another remote device as well as themovable object 1000. For example, the movable object 1000 and/orterminal 1012 may communicate with another movable object, or a carrieror payload of another movable object. When desired, the remote devicemay be a second terminal or other computing device (e.g., computer,laptop, tablet, smartphone, or other mobile device). The remote devicecan be configured to transmit data to the movable object 1000, receivedata from the movable object 1000, transmit data to the terminal 1012,and/or receive data from the terminal 1012. Optionally, the remotedevice can be connected to the Internet or other telecommunicationsnetwork, such that data received from the movable object 1000 and/orterminal 1012 can be uploaded to a website or server.

FIG. 11 is a schematic illustration by way of block diagram of a system1100 for controlling a movable object, in accordance with embodiments.The system 1100 can be used in combination with any suitable embodimentof the systems, devices, and methods disclosed herein. The system 1100can include a sensing module 1102, processing unit 1104, non-transitorycomputer readable medium 1106, control module 1108, and communicationmodule 1110.

The sensing module 1102 can utilize different types of sensors thatcollect information relating to the movable objects in different ways.Different types of sensors may sense different types of signals orsignals from different sources. For example, the sensors can includeinertial sensors, GPS sensors, proximity sensors (e.g., lidar), orvision/image sensors (e.g., a camera). The sensing module 1102 can beoperatively coupled to a processing unit 1104 having a plurality ofprocessors. In some embodiments, the sensing module can be operativelycoupled to a transmission module 1112 (e.g., a Wi-Fi image transmissionmodule) configured to directly transmit sensing data to a suitableexternal device or system. For example, the transmission module 1112 canbe used to transmit images captured by a camera of the sensing module1102 to a remote terminal.

The processing unit 1104 can have one or more processors, such as aprogrammable processor (e.g., a central processing unit (CPU)). Theprocessing unit 1104 can be operatively coupled to a non-transitorycomputer readable medium 1106. The non-transitory computer readablemedium 1106 can store logic, code, and/or program instructionsexecutable by the processing unit 1104 for performing one or more steps.The non-transitory computer readable medium can include one or morememory units (e.g., removable media or external storage such as an SDcard or random access memory (RAM)). In some embodiments, data from thesensing module 1102 can be directly conveyed to and stored within thememory units of the non-transitory computer readable medium 1106. Thememory units of the non-transitory computer readable medium 1106 canstore logic, code and/or program instructions executable by theprocessing unit 1104 to perform any suitable embodiment of the methodsdescribed herein. For example, the processing unit 1104 can beconfigured to execute instructions causing one or more processors of theprocessing unit 1104 to analyze sensing data produced by the sensingmodule. The memory units can store sensing data from the sensing moduleto be processed by the processing unit 1104. In some embodiments, thememory units of the non-transitory computer readable medium 1106 can beused to store the processing results produced by the processing unit1104.

In some embodiments, the processing unit 1104 can be operatively coupledto a control module 1108 configured to control a state of the movableobject. For example, the control module 1108 can be configured tocontrol the propulsion mechanisms of the movable object to adjust thespatial disposition, velocity, and/or acceleration of the movable objectwith respect to six degrees of freedom. Alternatively or in combination,the control module 1108 can control one or more of a state of a carrier,payload, or sensing module.

The processing unit 1104 can be operatively coupled to a communicationmodule 1110 configured to transmit and/or receive data from one or moreexternal devices (e.g., a terminal, display device, or other remotecontroller). Any suitable means of communication can be used, such aswired communication or wireless communication. For example, thecommunication module 1110 can utilize one or more of local area networks(LAN), wide area networks (WAN), infrared, radio, WiFi, point-to-point(P2P) networks, telecommunication networks, cloud communication, and thelike. Optionally, relay stations, such as towers, satellites, or mobilestations, can be used. Wireless communications can be proximitydependent or proximity independent. In some embodiments, line-of-sightmay or may not be required for communications. The communication module1110 can transmit and/or receive one or more of sensing data from thesensing module 1102, processing results produced by the processing unit1104, predetermined control data, user commands from a terminal orremote controller, and the like.

The components of the system 1100 can be arranged in any suitableconfiguration. For example, one or more of the components of the system1100 can be located on the movable object, carrier, payload, terminal,sensing system, or an additional external device in communication withone or more of the above. Additionally, although FIG. 11 depicts asingle processing unit 1104 and a single non-transitory computerreadable medium 1106, one of skill in the art would appreciate that thisis not intended to be limiting, and that the system 1100 can include aplurality of processing units and/or non-transitory computer readablemedia. In some embodiments, one or more of the plurality of processingunits and/or non-transitory computer readable media can be situated atdifferent locations, such as on the movable object, carrier, payload,terminal, sensing module, additional external device in communicationwith one or more of the above, or suitable combinations thereof, suchthat any suitable aspect of the processing and/or memory functionsperformed by the system 1000 can occur at one or more of theaforementioned locations.

As used herein A and/or B encompasses one or more of A or B, andcombinations thereof such as A and B.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A system for controlling an unmanned aerialvehicle within an environment, the system comprising: one or moresensors carried on the unmanned aerial vehicle and configured to receivesensor data of the environment; and one or more processors individuallyor collectively configured to: select, based on the sensor data, a firstset of operating rules for the unmanned aerial vehicle, receive userinput indicating a second set of operating rules for the unmanned aerialvehicle, select one of the first or second sets of operating rules to beused to control the unmanned aerial vehicle, receive a signal indicatinga desired movement of the unmanned aerial vehicle in said environment,and cause the unmanned aerial vehicle to move in accordance with thesignal while complying with the selected one of the first or second setsof operating rules.
 2. The system of claim 1, wherein the one or moresensors comprise at least one of the following: a vision sensor, a lidarsensor, or an ultrasonic sensor.
 3. The system of claim 1, wherein theone or more sensors comprise a plurality of different sensor types. 4.The system of claim 1, wherein the sensor data is indicative of anobstacle density for the environment.
 5. The system of claim 4, whereinthe one or more processors are configured to determine, based on thesensor data, an environmental complexity factor representative of theobstacle density for the environment.
 6. The system of claim 5, whereinthe first set of operating rules is selected based on the determinedenvironmental complexity factor.
 7. The system of claim 1, wherein theone or more processors are configured to generate a three-dimensionaldigital representation of the environment based on the sensor data. 8.The system of claim 1, wherein the first set of operating rulescomprises a first set of velocity rules and the second set of operatingrules comprises a second set of velocity rules.
 9. The system of claim8, wherein the second set of velocity rules is determined based on aflight mode selected by the user from a plurality of different flightmodes.
 10. The system of claim 9, wherein the plurality of differentflight modes comprises a low velocity flight mode, an intermediatevelocity flight mode, and a high velocity flight mode.
 11. The system ofclaim 8, wherein the first set of velocity rules comprises a firstvelocity limit for the unmanned aerial vehicle and the second set ofvelocity rules comprises a second velocity limit for the unmanned aerialvehicle.
 12. The system of claim 11, wherein the one or more processorsare configured to select the smaller of the first and second velocitylimits to be used to control the unmanned aerial vehicle.
 13. The systemof claim 8, wherein the first set of velocity rules constrains avelocity of the unmanned aerial vehicle based on whether an obstacledensity is relatively high or relatively low.
 14. The system of claim 1,wherein when the first and second set of operating rules are differentfrom each other, the one or more processors are configured to select theone of the first or second sets of operating rules that is less likelyto result in collisions with obstacles.
 15. The system of claim 1,wherein the user input is received from a remote terminal.
 16. A methodfor controlling an unmanned aerial vehicle within an environment, themethod comprising: receiving, with aid of a processor, sensor data ofthe environment from one or more sensors carried on the unmanned aerialvehicle; selecting, based on the sensor data and with aid of theprocessor, a first set of operating rules for the unmanned aerialvehicle; receiving, with aid of the processor, user input indicating asecond set of operating rules for the unmanned aerial vehicle;selecting, with aid of the processor, one of the first or second sets ofoperating rules to be used to control the unmanned aerial vehicle;receiving, with aid of the processor, a signal indicating a desiredmovement of the unmanned aerial vehicle in said environment; andcausing, with aid of the processor, the unmanned aerial vehicle to movein accordance with the signal while complying with the selected one ofthe first or second sets of operating rules.
 17. The method of claim 16,wherein the one or more sensors comprise at least one of the following:a vision sensor, a lidar sensor, or an ultrasonic sensor.
 18. The methodof claim 16, wherein the one or more sensors comprise a plurality ofdifferent sensor types.
 19. The method of claim 16, wherein the sensordata is indicative of an obstacle density for the environment.
 20. Themethod of claim 19, further comprising: determining, based on the sensordata and with aid of the processor, an environmental complexity factorrepresentative of the obstacle density for the environment.
 21. Themethod of claim 20, wherein the first set of operating rules is selectedbased on the determined environmental complexity factor.
 22. The methodof claim 16, further comprising: generating, with aid of the processor,a three-dimensional digital representation of the environment based onthe sensor data.
 23. The method of claim 16, wherein the first set ofoperating rules comprises a first set of velocity rules and the secondset of operating rules comprises a second set of velocity rules.
 24. Themethod of claim 23, wherein the second set of velocity rules isdetermined based on a flight mode selected by the user from a pluralityof different flight modes.
 25. The method of claim 24, wherein theplurality of different flight modes comprises a low velocity flightmode, an intermediate velocity flight mode, and a high velocity flightmode.
 26. The method of claim 23, wherein the first set of velocityrules comprises a first velocity limit for the unmanned aerial vehicleand the second set of velocity rules comprises a second velocity limitfor the unmanned aerial vehicle.
 27. The method of claim 26, furthercomprising: selecting the smaller of the first and second velocitylimits to be used to control the unmanned aerial vehicle.
 28. The methodof claim 23, wherein the first set of velocity rules constrains avelocity of the unmanned aerial vehicle based on whether an obstacledensity is relatively high or relatively low.
 29. The method of claim16, wherein when the first and second set of operating rules aredifferent from each other, the one of the first or second sets ofoperating rules that is less likely to result in collisions withobstacles is selected.
 30. The method of claim 16, wherein the userinput is received from a remote terminal.